Skip to content

improvement(chat): reduce streaming render work and completion delay - #8220

Merged
waleedlatif1 merged 2 commits into
stagingfrom
codex/streaming-replay-performance
Sep 24, 2026
Merged

improvement(chat): reduce streaming render work and completion delay#8220
waleedlatif1 merged 2 commits into
stagingfrom
codex/streaming-replay-performance

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Sep 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

  • Retain unchanged highlighted code lines and fast-path plain paragraphs through the existing Markdown/rendering pipeline.
  • Preserve word fades and live pacing, publish at up to 60 updates/sec, and finish buffered text over a 400 ms completion horizon.
  • Local production measurements on synthetic 32 KiB responses show 70% less CPU for paragraphs, 46% for structured Markdown, and 65% for code; final reveal tails fell from about 2.7 seconds to 0.4 seconds. The PR contains only the rendering fixes and regression tests.

Type of Change

  • Performance improvement

Testing

  • Focused renderer, special-tag, file-preview, and pacing suites; differential parser/token tests and frame-cadence tests.
  • All workspace type checks, 49 repository audits (including strict API validation), block-registry audit, docs-manifest check, and lint.
  • Repeated Chromium production replays, ordinary-production cross-check, slower arrival rate, final screenshot comparisons, and live fade/code-copy checks.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@waleedlatif1
waleedlatif1 requested a review from a team as a code owner September 24, 2026 02:07
@vercel

vercel Bot commented Sep 24, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Sep 24, 2026 2:29am UTC

Request Review

@greptile-apps

greptile-apps Bot commented Sep 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge; no outstanding or newly introduced actionable issue was identified.

Summary

This PR reduces streaming chat rendering work while preserving Markdown semantics and paced text disclosure.

  • Reuses unchanged Prism-highlighted line nodes during code streaming.
  • Adds a conservative plain-paragraph parsing fast path.
  • Caps streaming publication at approximately 60 updates per second.
  • Drains buffered completion text over a fixed 400 ms horizon.
  • Adds differential rendering, parsing, cadence, and completion regression tests.
Diagram 
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Stream[Incoming chat content] --> Smooth[useSmoothText pacing]
  Smooth -->|Up to 60 updates/sec| Markdown[Streamdown Markdown pipeline]
  Markdown --> Plain{Plain paragraph?}
  Plain -->|Yes| Fast[Plain-text AST fast path]
  Plain -->|No| Parse[Standard remark parsing]
  Fast --> Render[Chat rendering]
  Parse --> Render
  Render --> Code{Code fence?}
  Code -->|Yes| Highlight[Prism highlighting]
  Highlight --> Lines[Retained highlighted line nodes]
  Code -->|No| Prose[Rendered prose]
Loading

Reviews (4) · Last reviewed commit: "chore(chat): align streaming constants w..."

cubic-dev-ai[bot]
cubic-dev-ai Bot reviewed Sep 24, 2026
View reviewed changes

@cubic-dev-ai cubic-dev-ai Bot left a comment •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 28 files

Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Fix all with cubic | Re-trigger cubic

Comment thread apps/sim/scripts/streaming-replay/results/slower.json Outdated
@waleedlatif1
waleedlatif1 force-pushed the codex/streaming-replay-performance branch from 957dd2e to 33e83ba Compare September 24, 2026 02:18
@waleedlatif1

waleedlatif1 commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

waleedlatif1 commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai[bot]
cubic-dev-ai Bot reviewed Sep 24, 2026
View reviewed changes

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 7 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

@waleedlatif1

waleedlatif1 commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

waleedlatif1 commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai[bot]
cubic-dev-ai Bot reviewed Sep 24, 2026
View reviewed changes

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 7 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

@waleedlatif1

waleedlatif1 commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

waleedlatif1 commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai[bot]
cubic-dev-ai Bot reviewed Sep 24, 2026
View reviewed changes

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 7 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit 58a2ed9 into staging Sep 24, 2026
34 checks passed
@waleedlatif1
waleedlatif1 deleted the codex/streaming-replay-performance branch September 24, 2026 02:36

This branch was previously deployed

1 inactive deployment
Preview — a545a90d Deployed Sep 24, 2026 by vercel[bot]
View deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@greptile-apps greptile-apps[bot] greptile-apps[bot] left review comments
@cubic-dev-ai cubic-dev-ai[bot] cubic-dev-ai[bot] left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@waleedlatif1